Simulating activities: Relating motives, deliberation, and attentive coordination

Activities are located behaviors, taking time, conceived as socially meaningful, and usually involving interaction with tools and the environment. In modeling human cognition as a form of problem solving (goal-directed search and operator sequencing), cognitive science researchers have not adequately studied “off-task” activities (e.g., waiting), non-intellectual motives (e.g., hunger), sustaining a goal state (e.g., playful interaction), and coupled perceptual-motor dynamics (e.g., following someone). These aspects of human behavior have been considered in bits and pieces in past research, identified as scripts, human factors, behavior settings, ensemble, flow experience, and situated action. More broadly, activity theory provides a comprehensive framework relating motives, goals, and operations. This paper ties these ideas together, using examples from work life in a Canadian High Arctic research station. The emphasis is on simulating human behavior as it naturally occurs, such that “working” is understood as an aspect of living. The result is a synthesis of previously unrelated analytic perspectives and a broader appreciation of the nature of human cognition. Simulating activities in this comprehensive way is useful for understanding work practice, promoting learning, and designing better tools, including human-robot systems

AI: Inventing a new kind of machine.

A means-ends approach to engineering an artificial intelligence machine now suggests that we focus on the differences between human capabilities and the best computer programs. These differences suggest two basic limitations in the "symbolic" approach. First, human memory is much more than a storehouse where structures are put away, indexed, and rotely retrieved. Second, human reasoning involves more than searching, matching, and recombining previously stored descriptions of situations and action plans. Indeed, these hypotheses are related: Remembering and reasoning both involve reconceptualization. This short paper outlines recent work in situated cognition, robotics, and neural networks that suggests we frame the problem if AI in terms of inventing a new kind of machine

Visualizing practical knowledge: The Haughton-Mars Project

To improve how we envision knowledge, we must improve our ability to see knowledge in everyday life. That is, visualization is concerned not only with displaying facts and theories, but also with finding ways to express and relate tacit understanding. Such knowledge, although often referred to as "common," is not necessarily shared and may be distributed socially in choreographies for working together—in the manner that a chef and a maitre d’hôtel, who obviously possess very different skills, coordinate their work. Furthermore, non-verbal concepts cannot in principle be inventoried. Reifying practical knowledge is not a process of converting the implicit into the explicit, but pointing to what we know, showing its manifestations in our everyday life. To this end, I illustrate the study and reification of practical knowledge by examining the activities of a scientific expedition in the Canadian Arctic—a group of scientists preparing for a mission to Mar

Modeling the perceptual component of conceptual learning—A coordination perspective

Although a picture may be worth a thousand words, modeling diagrams as propositions and modeling visual processing as search through a database of verbal descriptions obscures what is problematic for the learner. Cognitive modeling of language learning and geometry has obscured the learner's problem of knowing where to look—what spaces, markings, and orientations constitute the objects of interest? Today we are launching into widespread use of multimedia instructional technology, without an adequate theory to relate perceptual processes to conceptual learning. Does this matter? In this article, I review the symbolic approach to modeling perceptual processing and show its limitations for explaining difficulties children encounter in interpreting a graphic display. I present an alternative analysis by which perceptual categorization is coupled to behavior sequences, where gesturing and emotional changes are essential for resolving impasses and breaking out of loops. I conclude by asking what kind of cognitive theory we need to exploit communication technology. Have we been correct to assume that pedagogy must be grounded in an accurate psychological model of knowledge, memory, and learning

The Newell Test Should Commit to Diagnosing Dysfunctions

"Conceptual coordination" analysis bridges connectionism and symbolic approaches by posting a "process memory" by which categories are physically coordinated (as neural networks) in time. Focusing on dysfunctions and odd behaviors like slips reveals the function of consciousness, especially taken-for-granted constructive processes, different from conventional programming constructs. Newell strongly endorsed identifying architectural limits; the heuristic of "diagnose unusual behaviors" will provide targets of opportunity that greatly strengthens the Newell Test

Clear Speaking about Machines: People are Exploring Mars, Not Robots

The primary responsibility of all scientists is to ensure the integrity of their work. For cognitive and social scientists, this means first and foremost preserving clarity about what we know about people, and not allowing descriptions of technology to demean or obscure the reality of how people think, behave, and live. Without this clarity, engineering requirements analyses, tool design, and evaluations of people will be confused. A sharp, uncompromising understanding about the nature of people is essential if we are to design and fit new technologies that are appropriate and successful for NASA's mission operations

Cognitive modeling of social behaviors

To understand both individual cognition and collective activity, perhaps the greatest opportunity today is to integrate the cognitive modeling approach (which stresses how beliefs are formed and drive behavior) with social studies (which stress how relationships and informal practices drive behavior). The crucial insight is that norms are conceptualized in the individual mind as ways of carrying out activities. This requires for the psychologist a shift from only modeling goals and tasks —why people do what they do—to modeling behavioral patterns—what people do—as they are engaged in purposeful activities. Instead of a model that exclusively deduces actions from goals, behaviors are also, if not primarily, driven by broader patterns of chronological and located activities (akin to scripts). To illustrate these ideas, this article presents an extract from a Brahms simulation of the Flashline Mars Arctic Research Station (FMARS), in which a crew of six people are living and working for a week, physically simulating a Mars surface mission. The example focuses on the simulation of a planning meeting, showing how physiological constraints (e.g., hunger, fatigue), facilities (e.g., the habitat’s layout) and group decision making interact. Methods are described for constructing such a model of practice, from video and first-hand observation, and how this modeling approach changes how one relates goals, knowledge, and cognitive architecture. The resulting simulation model is a powerful complement to task analysis and knowledge-based simulations of reasoning, with many practical applications for work system design, operations management, and training

Are You Talking to Me? Dialogue Systems Supporting Mixed Teams of Humans and Robots

This position paper describes an approach to building spoken dialogue systems for environments containing multiple human speakers and hearers, and multiple robotic speakers and hearers. We address the issue, for robotic hearers, of whether the speech they hear is intended for them, or more likely to be intended for some other hearer. We will describe data collected during a series of experiments involving teams of multiple human and robots (and other software participants), and some preliminary results for distinguishing robot-directed speech from human-directed speech. The domain of these experiments is Mars-analogue planetary exploration. These Mars-analogue field studies involve two subjects in simulated planetary space suits doing geological exploration with the help of 1-2 robots, supporting software agents, a habitat communicator and links to a remote science team. The two subjects are performing a task (geological exploration) which requires them to speak with each other while also speaking with their assistants. The technique used here is to use a probabilistic context-free grammar language model in the speech recognizer that is trained on prior robot-directed speech. Intuitively, the recognizer will give higher confidence to an utterance if it is similar to utterances that have been directed to the robot in the past